Field of the Invention
This invention relates to testing software applications and has particular application in a software test environment where the software being tested has dependencies on many applications.
Description of the Related Art
Computing systems are comprised of many hardware and software components, and dependencies between software applications are now common. An application may depend on another application running on the same hardware enclosure, or on applications executing anywhere within a Local Area Network (LAN) or Wide Area Network (WAN). A thorough test of a software application and any interfaces or dependencies is important to the successful delivery of a software product. Conventional technology can require that the dependencies of an application be present in the system in order for the application to communicate with each of its dependencies.
In a production runtime environment the dependent applications are present and the maintenance of the dependencies is relatively low. Once the applications are loaded changes to the system are generally infrequent. For this reason a production system is a good testing environment. However, a production system running live code with real customers, and disrupting customers from doing their work due to a software bug can have a significant financial impact. A second issue with a production environment is that the production facilities may be geographically removed from the developer/testers or due to security reasons removed from developers and testers needing to test an application. A testing or debugging environment on the other hand is dynamic. Frequent changes in software and hardware are common. It is built and maintained specifically for the purpose of testing hardware and software. Access to a test system is generally more open than that of a production facility. In a test lab code can be instrumented using intrusive and non-intrusive tools. The system can be rebooted or restarted generally at will to debug or test an application.
A rigorous and exhaustive test, tests the logic paths, varying the inputs, validating the outputs, and testing the calls to and returns from dependent applications. A thorough test of the application may include testing under many different test scenarios such as: 1) initial power on, 2) rebooting, 3) reset, 4) power cycle, 5) hardware malfunction, and 6) software aborts to name a few.
For the same reasons that a test lab is a good test environment; the ability to frequently reboot, reset, and frequent hardware and software changes made at will make a test environment a difficult environment for testing. Due to the frequent reboots and code changes the testers need to assume that the dependencies and applications may not be running and therefore put checks in place to verify that the application dependencies are running. This can be an enormous task requiring many man-hours to verify that the dependencies are executing. If a dependency or application fails time is spent debugging the malfunction, and it may require special skills to determine the cause of the error; for example it may require someone with special knowledge of the dependent application to determine why the application failed. Due to the number of dependencies that a single application can have, determining the cause of the error can be very time consuming and resource intensive. In one implementation a developer writes scripts or programs to discover if the dependent application is loaded. While this automates the dependency discovery process, separate code must be written to check that each dependency is loaded and running before an application is tested. If a dependency is not running, a tester needs to investigate why.
Another issue that arises when testing is that the testers may not have access to all of the dependent applications, either due to licensing issues or the financial cost of acquiring the dependent applications for the testing environment.
Therefore, from the foregoing discussion, it should be apparent that a need exists for an improved testing tool. A test harness that can identify the dependent applications, input parameters and return values. A test harness that can present the software calls made by a target application to the dependent applications that the target application calls. Preferably, such a method would, in certain embodiments, include the capability to save and playback the same return value for a given set of input parameters.